A Simple Geometric Explanation of Occam’s Razor

Occam’s razor states that out of possible explanations, plans, and designs, we should select the simplest one. It turns out that in many practical situations, the simplest explanation indeed turns out to be the correct one, the simplest plan is often the most successful, etc. But why this happens is not very clear. In this paper, we provide a simple geometric explanation of Occam’s razor. 1 Formulation of the Problem Occam razor: reminder. The famous principle – attributed to a medieval philosopher William of Ockham – is that we should look for the simplest possible explanation to different phenomena, for the simplest possible plans and designs, etc. Metaphorically, if we have an explanation or a design that is too complicated, we should try to “cut off” unnecessary parts – just like a razor can cut off unnecessary parts of the beard. This is the reason why this principle is known as Occam’s razor. Occam razor is a very efficient idea. Interestingly, the Occam razor principle not only leads to a simple explanations and designs, but it also helps to find an explanation or a design in the first place; see, e.g., [1, 2, 3]. With the advent of computers, is this principle still important? In the past, before computers were invented and all the computations had to be performed largely by hand, having a simple easy-to-calculate model was a necessity. However, nowadays, with powerful computers readily available, it should not matter whether a model is slightly more complicated than necessary – computations are fast anyway. Yes, this principle is still important. It should not matter, but, surprisingly, it does matter: in many practical situations, when we find a way to sim-